Perceptive Presence

together, people have used them for communication. From email—one of the first network applications—to the original MIT Zephyr notification system to the now-popular AOL Instant Messenger System, people use computers to break down physical distances and provide a sense of awareness about another person in another place. Beyond basic text interaction, research—such as the Portholes project at Xerox PARC—has strived to bring people together by giving them the ability to, for example, share snapshots or videos of themselves in remote places. Although these systems use still shots or video, they still required the human to perform intelligent processing to determine whether people were present, busy, or available to talk. As interaction with computers moves away from the traditional domain of a fixed keyboard and mouse moving a pointer on a screen, it becomes possible for the computer itself to determine a person’s state using perceptive presence applications. This state could include basic information, such as location and head pose, but also more useful information such as whether the person is present or busy. These applications are perceptive because they provide an alternative to the traditional screen-based GUI, using physical presence and gestures to replace the traditional keyboard and mouse. They are also presence applications because they seek to share information about a person or location across some distance. For example, perceptive presence applications can share information about the number of people in an area as well as their location, activity level, or line of site (where they are looking). Applications involving communication, interaction with objects—such as light switches, telephones, or selections on a screen— in a room, or security can use this information. Currently, many approaches in this domain offer tightly integrated systems that receive information from sensors and process it internally in the application. This approach requires developing new vision modules, integration languages, and architectures for each new application. If multiple applications need to run at once, computation will be redundant, as each application would have to process the raw data from the camera into its own representations. A general approach with a common interface and reusable processing units would greatly reduce computational redundancy, allow for easy integration, and simplify the use of vision interfaces to the level of today’s GUIs. We present a general XML-based interface for perceptive presence applications and a set of widgets that use this interface to infer activity information about a place. The interface connects any number of computer vision modules to perceptually aware applications. Figure 1 shows the implementation’s architecture. Although we’ve interfaced our system only to computer vision widgets, there is no limitation to having other sensing technologies provide information to the widget layer through the interface. Examples could include microphones or touch sensors for locating activity areas.